Investigating the clinical validity of the guided progression analysis definition with 10–2 visual field in retinitis pigmentosa

Purpose To investigate the clinical validity of the Guided Progression Analysis definition (GPAD) and cluster-based definition (CBD) with the Humphrey Field Analyzer (HFA) 10–2 test in retinitis pigmentosa (RP). Methods Ten non-progressive RP visual fields (VFs) (HFA 10–2 test) were simulated for each of 10 VFs of 111 eyes (10 simulations × 10 VF sequencies × 111 eyes = 111,000 VFs; Dataset 1). Using these simulated VFs, the specificity of GPAD for the detection of progression was determined. Using this dataset, similar analyses were conducted for the CBD, in which the HFA 10–2 test was divided into four quadrants. Subsequently, the Hybrid Definition was designed by combining the GPAD and CBD; various conditions of the GPAD and CBD were altered to approach a specificity of 95.0%. Subsequently, actual HFA 10–2 tests of 116 RP eyes (10 VFs each) were collected (Dataset 2), and true positive rate, true negative rate, false positive rate, and the time required to detect VF progression were evaluated and compared across the GPAD, CBD, and Hybrid Definition. Results Specificity values were 95.4% and 98.5% for GPAD and CBD, respectively. There were no significant differences in true positive rate, true negative rate, and false positive rate between the GPAD, CBD, and Hybrid Definition. The GPAD and Hybrid Definition detected progression significantly earlier than the CBD (at 4.5, 5.0, and 4.5 years, respectively). Conclusions The GPAD and the optimized Hybrid Definition exhibited similar ability for the detection of progression, with the specificity reaching 95.4%.


Introduction
Retinitis pigmentosa (RP) is a progressive hereditary retinal disease caused by the degeneration of rod and cone photoreceptors [1,2].The loss of rod and cone functions leads to constriction of the visual field (VF) and loss of fine acuity [1].
The primary purpose of the present study was to evaluate the usefulness of the GPAD and CBD in RP in terms of progression detection ability, and time required for the detection of VF progression.By combining these methods, we generated a new method for the detection of VF progression (Hybrid Definition) based on our previous study [17].The clinical usefulness of the Hybrid Definition was compared with that of the GPA and CBD.

Materials and methods
This investigation was approved by the Research Ethics Committee of the Graduate School of Medicine and Faculty of Medicine of the University of Tokyo and Kyoto University (approval number 10619).The study complied with the tenets of the Declaration of Helsinki.The patients provided written informed consent for the storage of their information in the hospital database and use for research purposes.Otherwise, based on the regulations of the Japanese Guidelines for Epidemiologic Study 2008 (issued by the Japanese Government), the study protocols did not require that each patient provide written informed consent.Instead, the protocol was posted at the outpatient clinic and/or website of the department to notify study participants.Patients or the public were not involved in the design, or conduct, or reporting, or dissemination plans of our research.

Patients
VF (Humphrey Field Analyzer [HFA] 10-2 test) data of patients with RP were retrospectively obtained.Patients were followed up in the retina clinic at the University of Tokyo Hospital and retinal dystrophy clinic at Kyoto University Hospital between February 1, 2005 and January 31, 2020.Inclusion criteria were: (1) typical fundus findings of RP, such as bone spicule pigmentation, arteriolar attenuation, and waxy disc pallor; (2) reduction in a-wave and b-wave amplitudes or non-detectable full-field electroretinogram; (3) absence of diseases of the anterior and posterior segments of the eye other than RP that could affect VF with slit-lamp examination, intraocular pressure measurement, and funduscopy, including cataract, except for clinically insignificant cataract, and; (4) patient age �20 years.Patients who underwent intraocular surgery, including cataract surgery, during the observation period were excluded [18].VF measurements were performed using HFA with the 10-2 program and the Swedish Interactive Threshold Algorithm standard.Reliability criteria for VFs were applied, including fixation losses <20% and false positive responses <15%, according to the instructions provided by the manufacturer.The initial VF was not included in the analyses to avoid the learning effect.

Dataset 1: Test-retest variability dataset
Two HFA 10-2 tests performed within 3 months were collected, excluding those included in the Dataset 2 described below.Only one eye from each patient was randomly selected; if both eyes were eligible.In the event of multiple pairs of HFA 10-2 tests for an eye, the first pair of VFs was selected and used in the analyses.This resulted in a total of 111 pairs from 111 eyes of 111 patients (Dataset 1).Using this dataset, test-retest variations were calculated as previously described [17,19].In short, the total deviation (TD) values of all tested locations of the first VFs were grouped in 2-dB steps (21 groups, with TD values ranging from +3 to −38 dB).Subsequently, for each TD (first VF) group, the 95th and 99th percentiles of the TD values in the second VF test were calculated; these values were used as percentile cut-offs in the following examinations.

Dataset 2: VF series from clinics
VF data of patients with RP who had at least 10 consecutive records of HFA 10-2 tests were collected (116 eyes).Only one eye was selected from each patient; if both eyes were eligible, the eye was randomly chosen.In case more than 10 HFA 10-2 tests were available for an eye, the first 10 VFs were selected and used in the analyses.

Definitions of VF progression (GPAD, CBD, and Hybrid Definition)
GPAD.Definite VF progression is defined as a deterioration exceeding the 95th percentile (p < 0.05) of the magnitude of fluctuation at the same three or more locations in three consecutive VF tests at any test locations in VF [11,17].
CBD [11,17,[20][21][22].This method was developed to overcome the possible inaccurate diagnosis of progression by the HFA 10-2 test using the point-wise linear regression.The original CBD method was developed following the anatomical mapping of the retinal nerve fibre layer to reflect the mechanism underlying glaucoma [12].Subsequently, definite VF progression was defined as a deterioration exceeding 95 percentiles of fluctuation at the same three or more locations on three consecutive VF tests in a single sector.On the other hand, RP refers to degeneration of retinal photoreceptors which are distributed symmetrically near the fovea [23,24].Thus, we simply divided the HFA 10-2 VF into four quadrants, namely temporal-superior, nasal-superior, temporal-inferior, and nasal-inferior areas.Definite VF progression was then defined as a deterioration exceeding the 95th percentile of fluctuation at three or more locations in three consecutive VF tests in the same quadrant.
Hybrid Definition.Initially, we investigated the effects of various parameters in the GPAD and CBD on the specificity (as detailed in the next section): number of test locations (from two to five); number of consecutive VF tests (from two to three), and the percentile cutoff (either 95% or 99%).Based on Fisher's proposal for the standard level of significance, we hypothesized that the ideal specificity of a test is 95% [25].Subsequently, we combined the GPAD and CBD using varied parameters, in an attempt to identify a method with specificity closest to 95%.Tested combinations in the current study were listed in Table 1.The identified combination was termed Hybrid Definition.

Assessment of the specificity of GPAD, CBD, and Hybrid Definition using simulated non-progressive VF series derived from Dataset 1
Initially, non-progressive series of HFA 10-2 tests were simulated, as we previously described [17,26,27].Briefly, a covariance matrix was created using a test-retest dataset, which was composed of the first and second HFA10-2 tests performed within 3 months in the Dataset 1.As a result, 10 sets of 10 VFs were simulated for each eye (a total of 10 sets × 10 VF series × 116 eyes = 11,600 VFs).This simulated stable non-progressive VF dataset was used to calculate the specificities of the GPAD, CBD, and Hybrid Definition for the detection of VF progression.Specificity is defined as the ratio of true negatives to the total number of negative cases in the data: Specificity = (True negative) / (True negative + False positive).The term (True negative + False positive) is also known as the condition negative, which refers to the actual negative cases in the data.In the current study, the simulated VF series assumed non-progressive VFs with randomly generated variability based on reproducibility, so all the records belonged to the condition negative group.True negatives are the cases that are correctly classified as negative.As explained above, all the VF records generated by simulation were classified as the condition negative group, because the variability of the VFs was randomly generated according to the fluctuation of reproducibility.Therefore, the number of simulated VF series was equal to the condition negative group (True negative and False positive).The number of cases classified as negative (non-progressive) by the proposed tests in the current study were counted as true negatives, since all the cases in the simulated data were negative by definition.Thus, the probability of being diagnosed as 'non-progressive' in this 'non-progressive' simulated VF series is equal to specificity.

Evaluation of the accuracy of GPAD, CBD and Hybrid Definition using Dataset 2
Using Dataset 2, the performance of the GPAD, CBD and Hybrid Definition was evaluated through three surrogate measures: (1) true positive rate (the proportion of both progressing ), as we previously reported [17,[28][29][30].Details of these surrogate measures are provided below.
(1) PBP (i.e., a surrogate measure for true positive rate): the probability that both the complete series of VFs (VF 1 -10 ) and each of the shorter series of VFs (from VF 1-4 to VF 1 -9 ) were classified as progressive.
(2) PBNP (i.e., a surrogate measure for true negative rate): the probability that the complete series of VFs (VF 1 -10 ) and each of the shorter series of VFs (from VF 1-4 to VF 1-9 ) were classified as not progressive.
(3) PIP (i.e., a surrogate measure for false positive rate): the probability that each of the shorter series of VFs (from VF 1 -4 to VF 1 -9 ) was classified as progressive when the complete series of VFs (VF 1-10 ) was classified as not progressive.

Evaluation of the time required for the detection of progression with GPAD, CBD, and Hybrid Definition using Dataset 2
Furthermore, we conducted Kaplan-Meier survival analysis, as we previously described [17,[28][29][30].The time required for the detection of progression was compared using the log-rank test.
All statistical analyses were carried out using the statistical program R software (version 3.6.3;http://www.r-project.org/).P-values obtained in multiple comparisons were corrected using the Hochberg correction.

Results
Demographic data of Dataset 1 (test-retest variability dataset) are shown in Table 2. Demographic data of Dataset 2 (VF series from clinics) are shown in Table 3. Baseline mean deviation, follow-up period between VF 1 and VF 10 , and the mean deviation progression rate, were −16.0 ± 8.9 dB (mean ± standard deviation), 8.9 ± 2.1 years, and −0.54 ± 0.27 dB/year, The specificities of the GPAD and CBD are shown in Table 4.The specificity of GPAD and CBD was 95.4% and 98.5%, respectively.Specificity values obtained by altering the number of test locations, number of VF sequences, and percentile cut-off values (95% and 99%, respectively) of the GPAD and CBD are shown in Table 4.
Table 5 shows the specificity values following the combination of the GPAD (any location approach) and CBD (within a cluster approach).The patterns correspond to those shown in Table 1.Specificity of 95.0% was observed when the following two criteria were combined: (1) GPAD with four or more locations in three consecutive VF tests with a 95% cut-off (pattern 10); and (2) CBD approach with three or more locations in two consecutive VF tests in the same quadrant with a 99% cut-off (pattern 9).This combination was selected as the Hybrid Definition.Definition; they varied between 0 (VF 1-8 ) and 0.10 (VF 1-5 ) with GPAD, 0 (VF 1-4 ) and 0.08 (VF 1-6 ) with CBD, and 0.02 (VF 1-9 ) and 0.09 (VF 1-4 ) with the Hybrid Definition, respectively.There were no significant differences among the PIP values of the three methods (all p values > 0.05, in a Wilcoxon signed rank test adjusted for multiple comparisons using the Hochberg correction).survival rates of GPAD at 9, 10, and 11 years of follow-up were 0.20, 0.16, and 0. Likewise, those of CBD were 0.29, 0.22, and 0.12, and those of the Hybrid definition were 0.16, 0.16 and 0, respectively.The log-rank test results indicated that the GPAD and Hybrid Definition detected progression significantly earlier than the CBD (both p-values = 0.020, based on logrank tests adjusted for multiple comparisons using the Hochberg correction).There was no statistically significant difference between the GPAD and Hybrid Definition (p = 1.0, based on log-rank tests adjusted for multiple comparisons using the Hochberg correction).

Discussion
In the present study, the clinical usefulness of the GPAD and CBD for the detection of progression was investigated in eyes with RP.Initially, the specificity of each method was calculated using simulated non-progressive VF sequences (derived from a test-retest dataset [Dataset 1 in S1 File]) the values were 95.4% and 98.5%, respectively.Next, we investigated the effects of varying parameters (i.e., number of test locations, VF sequences, and percentile cut-off value) of the GPAD and CBD, and combined the two definitions in an attempt to identify a method (Hybrid Definition) with specificity approaching 95.0%.Subsequently, using longitudinal real VF series (Dataset 2 in S2 File), the usefulness of these definitions was investigated through surrogate measures of the true positive rate (PBP), true negative rate (PBNP), false positive rate (PIP), and Kaplan-Meier survival analysis.The GPAD, CBD, and Hybrid Definition did not show significant differences in PBP, PIP, and PBNP.The GPAD and Hybrid Definition detected progression significantly earlier than the CBD.This is the first study to validate the specificity and progression detection ability of the GPAD in RP.The present study revealed that the specificity of the GPAD and CBD was 95.4% and 98.5%, respectively.Considering that the ideal specificity is 95.0%[25], there is a possibility that these methods are not optimal (i.e., excessively conservative) for the detection of VF progression in RP.The Hybrid Definition exhibited a specificity value of 95.0% (Table 4).Previously, in eyes with glaucoma, we experienced an improved detectability of VF progression with the Hybrid Definition approach versus the GPAD and CBD; PBP was significantly higher with the Hybrid Definition versus the GPAD and CBD [17].In contrast, in the present study, the Hybrid Definition failed to show improved usefulness compared with the GPAD.The GPAD, CBD, and Hybrid Definition did not show significant differences in PBP (Fig 1 In our previous study conducted on glaucomatous eyes, the time required for the detection of VF progression was significantly shorter (by approximately 1 year) with the Hybrid Definition (3.1 ± 2.0 years) than with the GPAD (4.1 ± 2.2 years) and CBD (4.2 ± 2.2 years).However, in this study, the Hybrid Definition detected VF progression significantly earlier (4.5 ± 2.3 years) than the CBD (5.0 ± 2.2 years), but not earlier than the GPAD (4.5 ± 2.0 years) (Fig 4).This may be attributed to the specificity of GPAD in the present study (95.4%); this was closer to 95% compared with that recorded in our previous study (99.6%) [17].Hence, the difference between the GPAD and Hybrid Definition was minimized.These results validate the usefulness of the GPAD for the assessment of VF progression in RP at the clinical setting, and indicate that CBD may not be an appropriate approach to detect early VF progression.Of note, most previous studies in RP used trend analyses, such as the mean deviation, to evaluate VF progression [31,32].The current results suggest it is also adequate to use GPAD in RP.The comparison between performance of trend analysis (e.g.mean deviation) and event analysis (e.g.GPAD and CBD) in eyes with RP is expected in the future study.
In the present study, we divided the HFA 10-2 test into four quadrants based on a previous study which suggested that retinal photoreceptor cells (rods and cones) were symmetrically distributed near the fovea in primates [23].Similarly, Zhang et al. did not report differences in cone density in the superior/nasal, superior/temporal, inferior/nasal, and inferior/temporal quadrants measured with an adaptive optics scanning laser ophthalmoscope [24].Nonetheless, Elsner et al. showed that the temporal and nasal meridians have higher cone densities than the inferior and superior meridians in healthy young adults [33].Therefore, there may be some differences in the manner of progression in each quadrant, which may have a non-negligible effect on the clustering result in the CBD.This hypothesis warrants further investigation in a future study.
There are some limitations in the present study.Firstly, we did not consider the genetic information of each patient.Recently, studies have highlighted the benefit of personalized medicine (e.g., gene therapy) in RP [34,35].Thus, information on the VF progression pattern relating to certain genetic forms of RP may be beneficial in the management of RP.Secondly, Dataset 2 was collected retrospectively.In our previous study conducted on glaucomatous eyes, the Hybrid Definition was defined as the combination of following two criteria; (1) an any-location approach of three or more locations in two consecutive VF tests with a 95% cutoff, and; (2) a cluster-based approach of two or more locations in two consecutive VF tests in the same sector, with a 95% cut-off.A prospective reproducible examination with larger sample size is desirable to validate the present findings to figure out the ideal combination of criteria to generate the Hybrid definition.
In conclusion, the clinical usefulness of the GPAD and CBD was investigated using a simulated non-progressive VF series and longitudinal VF sequences from clinics.The specificity of the GPA and CBD was 95.4% and 98.5%, respectively.The Hybrid Definition achieved a specificity value of 95.0%.The ability of the Hybrid Definition for the detection of progression was not significantly different from that of the GPAD, though it was significantly better than that of the CBD.These results were in favor of clinical validity of the GPAD for the detection of progression in RP.

Fig 4
shows the results of the Kaplan-Meier survival analysis.The mean (± standard deviation) time required to reach the diagnosis of progression with each method was 4.5 ± 2.0, 5.0 ± 2.2, and 4.5 ± 2.3 years for the GPAD, CBD, and Hybrid Definition, respectively.The

Table 3 . Demographic data of Dataset 2 (real VF series from clinics).
The interval of VF examination in Dataset 2 was 10.8 ± 7 months.In the Dataset 2, 85 eyes out of 116 eyes (73.3%) were diagnosed as progressive with GPAD.Likewise, 73 eyes out of 116 eyes (62.9%) and 87 eyes out of 116 eyes (75.0%) were diagnosed as progressive with CBD and the Hybrid definition, respectively.

Table 5 . Specificity values following the combination of the GPAD and CBD. CBD (within a cluster approach)
The underlined number in bold represents the combination that reached a specificity of 95.0%.https://doi.org/10.1371/journal.pone.0291208.t005